y y ˆ i i Difference between
|
|
- Florence Franklin
- 5 years ago
- Views:
Transcription
1 Fttg Equatos Idea: Te varale of terest (depedet varale, y ) s ard to measure. Tere are easy to measure varales (predctor/ depedet) tat are related to te varale of terest, laeled,,... m We measure te y ad te s for a sample ad use ts sample to ft a model. Oce te model s ftted, we ca te just measure te s, ad get a estmate of y wtout measurg t Types of Equatos Smple Lear Equato: y β o + β + ε Multple Lear Equato: y β + β + β +...+β m m +ε Nolear Equato: takes may forms, for eample: Eample: Tree Hegt (m) ard to measure; D (dameter at. m aove groud cm) easy to measure use D squared for a lear equato e g t y y Dfferece etwee measured y ad te mea of y y yˆ D squared ˆ Dfferece etwee measured y ad predcted y y y ˆ y y y ˆ ( y y) ( y yˆ ) y Dfferece etwee predcted y ad mea of y y y ˆ y ˆ y y β + β β β +ε
2 Ojectve: Fd estmates of β, β, β... β m suc tat te sum of squared dffereces etwee measured y ad predcted y (usually laeled as ŷ, values o te le or surface) s te smallest (mmze te sum of squared errors, called least squared error). OR Fd estmates of β, β, β... β m suc tat te lkelood (proalty) of gettg tese y values s te largest (mamze te lkelood). Fdg te mmum of sum of squared errors s ofte easer. I some cases, tey lead to te same estmates of parameters. Least Squares Soluto: Fdg te Set of Coeffcets tat Mmzes te Sum of Squared Errors To fd te estmated coeffcets tat mmzes SSE for a partcular set of sample data ad a partcular equato (form ad varales):. Defe te sum of squared errors (SSE) terms of te measured mus te predcted y s (te errors);. Take partal dervatves of te SSE equato wt respect to eac coeffcet. Set tese equal to zero (for te mmum) ad solve for all of te equatos (solve te set of equatos usg algera or lear algera). 4
3 Smple Lear Regresso Propertes of ad Tere s oly oe varale Tere wll e two coeffcets Te estmated tercept s foud y: ad are least squares estmates of β ad β. Uder assumptos cocerg te error term ad samplg/ measuremets, tese are: Uased estmates; gve may estmates of te slope ad tercept for all possle samples, te average of te sample estmates wll equal te true values y Ad te estmated slope s foud y: ( y y)( ) ( ) s s y ( ) ( ) SPy SS Te varalty of tese estmates from sample to sample ca e estmated from te sgle sample; tese estmated varaces wll e uased estmates of te true varaces (ad stadard errors) Te estmated tercept ad slope wll e te most precse (most effcet wt te lowest varaces) estmates possle (called Best ) Tese wll also e te mamum lkelood estmates of te tercept ad slope Were SPy refers to te corrected sum of cross products for ad y; SS refers to te corrected sum of squares for [Class eample] 5 6
4 Assumptos of SLR Oce coeffcets are otaed, we must ceck te assumptos of SLR. Assumptos must e met to: ota te desred caracterstcs assess goodess of ft (.e., ow well te regresso le fts te sample data) test sgfcace of te regresso ad oter ypoteses calculate cofdece tervals ad test ypotess for te true coeffcets (populato) calculate cofdece tervals for mea predcted y value gve a set of value (.e. for te predcted y gve a partcular value of te ) Need good estmates (uased or at least cosstet) of te stadard errors of coeffcets ad a kow proalty dstruto to test ypoteses ad calculate cofdece tervals. Ceckg te followg assumptos usg resdual Plots. a lear relatosp etwee te y ad te ;. equal varace of errors across te rage of te y varales; ad. depedece of errors (depedet oservatos), ot related tme or space. A resdual plot sows te resdual (.e., y - ŷ ) as te y-as ad te predcted value ( ŷ ) as te -as. Resdual plots ca also dcate uusual pots (outlers) tat may e measuremet errors, trascrpto errors, etc. 7 8
5 Eamples of Resdual Plots Idcatg Falures to Meet Assumptos:. Te relatosp etwee te s ad y s lear. If ot. Te varace of te y values must e te same for every oe of te values. If ot met, te spread aroud te le wll ot e eve. met, te resdual plot ad te plot of y vs. wll sow a curved le: [CRITICAL ASSUMPTION!!] t 6 ˆ 5 ˆ 4 ˆ ˆ 4 4 ˆ ˆ 4 ˆ Šˆ ˆ ˆ ˆ ˆ ˆ dsq Resdual 5 ˆ * * * * * * ˆ * * *** * *** *** *** ** ** * *** ***** *** *** * *** * 5 ˆ ********* ** ******* * **** * ** ** * ****** *** ** * ***** ** * ** *** * * * ***** ** * * * **** ˆ ******** ** *** ******** * * * * ** ******* * * ******* ** * * * ******* * * * -5 ˆ * *** * * * * * * ** ** * * ** * ** * * * ** ** - ˆ * * * * * * * -5 ˆ * * - ˆ Š-ˆ ˆ ˆ ˆ ˆ ˆ ˆ Predcted Value of t Result: If ts assumpto s ot met, te estmated coeffcets (slopes ad tercept) wll e uased, ut te estmates of te stadard devato of tese coeffcets wll e ased. we caot calculate CI or test te sgfcace of te Result: If ts assumpto s ot met: te regresso le does ot ft te data well; ased estmates of coeffcets varale. However, estmates of te coeffcets of te regresso le ad goodess of ft are stll uased ad stadard errors of te coeffcets wll occur 9
6 . Eac oservato (.e., ad y ) must e depedet of all oter oservatos. I ts case, we produce a dfferet resdual plot, were te resduals are o te y-as as efore, ut te -as s te varale tat s tougt to produce te depedeces (e.g., tme). If ot met, ts revsed resdual plot wll sow a tred, dcatg te resduals are ot depedet. Result: If ts assumpto s ot met, te estmated coeffcets (slopes ad tercept) wll e uased, ut te estmates of te stadard devato of tese coeffcets wll e ased. we caot calculate CI or test te sgfcace of te varale. However, estmates of te coeffcets of te regresso le ad goodess of ft are stll uased Normalty Hstogram or Plot A fourt assumpto of te SLR s: 4. Te y values must e ormally dstruted for eac of te values. A stogram of te errors, ad/or a ormalty plot ca e used to ceck ts, as well as tests of ormalty Hstogram # Boplot.5+*.*.*.*.**** 8.******* 4.************** 7.******************** 4.***************************** ************************** 5.****************************** 6 *--+--* -.5+***************************** 58.************************* 49.***************** ************** 8.************ 4.***********.**** 7.**** 7.*** 5..* -.5+** HO: resduals are ormal H: resduals are ot ormal Tests for Normalty Test --Statstc p Value Sapro-Wlk W.99 Pr < W.9 Kolmogorov-Smrov D.98 Pr > D.67 Cramer-vo Mses W-Sq.96 Pr > W-Sq.66 Aderso-Darlg A-Sq.986 Pr > A-Sq <.5
7 Normal Proalty Plot.5+ * * +** +++** +**** +**** ***** **** ***** **** **** -.5+ **** ***+ **** *** +*** ***** +** +*** +**** * -.5+* Result: We caot calculate CI or test te sgfcace of te varale, sce we do ot kow wat proaltes to use. Also, estmated coeffcets are o loger equal to te mamum lkelood soluto. Resdual Frequecy Normal Plot of Resduals Normal Score Hstogram of Resduals Resdual Volume versus d 4 Resdual Resdual I Cart of Resduals Oservato Numer 5 Ft Resduals vs. Fts 6 5 UCL.6 X. LCL-.6 4
8 Measuremets ad Samplg Assumptos Te remag assumptos are ased o te measuremets ad collecto of te samplg data. 5. Te values are measured wtout error (.e., te values are fed). Ts ca oly e kow f te process of collectg te data s kow. For eample, f tree dameters are very precsely measured, tere wll e lttle error. If ts assumpto s ot met, te estmated coeffcets (slopes ad tercept) ad ter varaces wll e ased, sce te values are varyg. 6. Te y values are radomly selected for value of te varales (.e., for eac value, a lst of all possle y values s made, ad some are radomly selected). Ofte, te oservatos wll e gatered usg systematc samplg (grd across te lad area). Ts does ot strctly meet ts assumpto. Also, more comple samplg desg suc as multstage samplg (samplg large uts ad samplg smaller uts wt te large uts), ts assumpto s ot met. If te equato s correct, te ts does ot cause prolems. If ot, te estmated equato wll e ased. Trasformatos Commo Trasformatos Powers,.5, etc. for relatosps tat look olear log, loge also for relatosps tat look olear, or we te varaces of y are ot equal aroud te le S- [arcse] we te depedet varale s a proporto. Rak trasformato: for o-ormal data o Sort te y varale o Assg a rak to eac varale from to o Trasform te rak to ormal (e.g., Blom Trasformato) PROBLEM: loose some of te formato te orgal data Try to trasform frst ad leave y varale of terest; owever, ts s ot always possle. Use graps to elp coose trasformatos 5 6
9 Outlers: Uusual Pots Ceck for pots tat are qute dfferet from te oters o: Grap of y versus Resdual plot Do ot delete te pot as t MAY BE VALID! Ceck: Is ts a measuremet error? E.g., a tree egt of m s very ulkely Is a trascrpto error? E.g. for adult perso, a wegt of ls was etered rater ta ls. Is tere sometg very uusual aout ts pot? e.g., a rd as a sort eak, ecause t was damaged. Try to f te oservato. If t s very dfferet ta te oters, or you kow tere s a measuremet error tat caot e fed, te delete t ad dcate ts your researc report. O te resdual plot, a outler CAN occur f te model s ot correct may eed a trasformato of te varale(s), or a mportat varale s mssg Measures of Goodess of Ft How well does te regresso ft te sample data? For smple lear regresso, a grap of te orgal data wt te ftted le marked o te grap dcates ow well te le fts te data [ot possle wt MLR] Two measures commoly used: coeffcet of determato (r ) ad stadard error of te estmate(se E ). To calculate r ad SE E, frst, calculate te SSE (ts s wat was mmzed): SSE e ( y yˆ ) ( y ( + )) Te sum of squared dffereces etwee te measured ad estmated y s. Calculate te sum of squares for y: ( ) SSy y y y y sy ( ) Te sum of squared dfferece etwee te measured y ad te mea of y-measures. NOTE: I some tets, ts s called te sum of squares total. 7 8
10 Calculate te sum of squares regresso: SSreg ( y yˆ ) SPy SSy SSE Te sum of squared dffereces etwee te mea of y- measures ad te predcted y s from te ftted equato. Also, s te sum of squares for y te sum of squared errors. SSy SSE SSE SSreg Te: r SSy SSy SSy SSE, SSY are ased o y s used te equato wll ot e orgal uts f y was trasformed r coeffcet of determato; proporto of varace of y, accouted for y te regresso usg Is te square of te correlato etwee ad y O (very poor orzotal surface represetg o relatosp etwee y ad s) to (perfect ft surface passes troug te data) Ad: SSE SE E SSE s ased o y s used te equato wll ot e orgal uts f y was trasformed SE E - stadard error of te estmate; same uts as y Uder ormalty of te errors: o ± SE E 68% of sample oservatos o ± SE E 95% of sample oservatos o Wat low SEE 9
11 y-varale was trasformed: Ca calculate estmates of tese for te orgal y-varale ut, called I (Ft Ide) ad estmated stadard error of te estmate (SE E ), order to compare to r ad SE E of oter equatos were te y was ot trasformed. I - SSE/SSY were SSE, SSY are orgal uts. NOTE must ack-trasform te predcted y s to calculate te SSE orgal uts. Does ot ave te same propertes as r, owever: o t ca e less ta o t s ot te square of te correlato etwee te y ( orgal uts) ad te used te equato. Estmated stadard error of te estmate (SE E ), we te depedet varale, y, as ee trasformed: SSE( orgal uts) SE E ' SE E - stadard error of te estmate ; same uts as orgal uts for te depedet varale wat low SE E [Class eample] Estmated Varaces, Cofdece Itervals ad Hypotess Tests Testg Weter te Regresso s Sgfcat Does kowledge of mprove te estmate of te mea of y? Or s t a flat surface, wc meas we sould just use te mea of y as a estmate of mea y for ay? SSE/ (-): Called te Mea squared error, as would e te average of te squared error f we dvded y. Istead, we dvde y -. Wy? Te degrees of freedom are -; oservatos wt two statstcs estmated from tese, ad Uder te assumptos of SLR, s a uased estmated of te true varace of te error terms (error varace) SSR/: Called te Mea Square Regresso Degrees of Freedom: -varale Uder te assumptos of SLR, ts s a estmate te error varace PLUS a term of varace eplaed y te regresso usg.
12 H: Regresso s ot sgfcat H: Regresso s sgfcat Same as: H: β [true slope s zero meag o relatosp wt ] H: β [slope s postve or egatve, ot zero] Ts ca e tested usg a F-test, as t s te rato of two varaces, or wt a t-test sce we are oly testg oe coeffcet (more o ts later) Iformato for te F-test s ofte sow as a Aalyss of Varace Tale: Source df SS MS F p-value Regresso MSreg F Pro F> SSreg SSreg/ MSreg/MSE F (,-,- α) Resdual - SSE MSE SSE/(-) Total - SSy [Class eample ad eplaato of te p-value] Usg a F test statstc: SSreg F SSE ( ) MSreg MSE Uder H, ts follows a F dstruto for a - α/ percetle wt ad - degrees of freedom. If te F for te ftted equato s larger ta te F from te tale, we reject H (ot lkely true). Te regresso s sgfcat, tat te true slope s lkely ot equal to zero. 4
13 Estmated Stadard Errors for te Slope ad Itercept Uder te assumptos, we ca ota a uased estmated of te stadard errors for te slope ad for te tercept [measure of ow tese would vary amog dfferet sample sets], usg te oe set of sample data. s s MSE + SS MSE SS MSE SS Cofdece Itervals for te True Slope ad Itercept Uder te assumptos, cofdece tervals ca e calculated as: For β o : ± t α, s Hypotess Tests for te True Slope ad Itercept H: β c [true slope s equal to te costat, c] H: β c [true slope dffers from te costat c] Test statstc: t c s Uder H, ts s dstruted as a t value of t c t -, -α/. Reject H o f t > t c. Te procedure s smlar for testg te true tercept for a partcular value It s possle to do oe-sded ypoteses also, were te alteratve s tat te true parameter (slope or tercept) s greater ta (or less ta) a specfed costat c. MUST e careful wt te t c as ts s dfferet. [class eample] t s ± For β : α, [class eample] 5 6
14 Cofdece Iterval for te True Mea of y gve a partcular value For te mea of all possle y-values gve a partcular value of (μ y ): were yˆ s yˆ ˆ y ± t, α syˆ + MSE + ( ) SS Cofdece Bads Plot of te cofdece tervals for te mea of y for several -values. Wll appear as: Predcto Iterval for or more y-values gve a partcular value For oe possle ew y-value gve a partcular value of : Were yˆ s ˆ y ( ew) ± t, α syˆ ( ew) ( ew) yˆ ( ew) + MSE + + ( ) SS For te average of g ew possle y-values gve a partcular value of : were yˆ s ( ew) ˆ y ( ew) ± t, α syˆ ( ewg ) yˆ ( ew g ) [class eample] + MSE + + g ( ) SS 7 8
15 Selectg Amog Alteratve Models Process to Ft a Equato usg Least Squares Steps:. Sample data are eeded, o wc te depedet varale ad all eplaatory (depedet) varales are measured.. Make ay trasformatos tat are eeded to meet te most crtcal assumpto: Te relatosp etwee y ad s lear. Eample: volume β + β d may e lear wereas volume versus d s ot. Use y volume, d.. Ft te equato to mmze te sum of squared error. 4. Ceck Assumptos. If ot met, go ack to Step. 5. If assumptos are met, te terpret te results. Is te regresso sgfcat? Wat s te r? Wat s te SE E? Plot te ftted equato over te plot of y versus. For a umer of models, select ased o:. Meetg assumptos: If a equato does ot meet te assumpto of a lear relatosp, t s ot a caddate model. Compare te ft statstcs. Select ger r (or I ), ad lower SE E (or SE E ). Reject ay models were te regresso s ot sgfcat, sce ts model s o etter ta just usg te mea of y as te predcted value. 4. Select a model tat s ologcally tractale. A smpler model s geerally preferred, uless tere are practcal/ologcal reasos to select te more comple model 5. Cosder te cost of usg te model [class eample] 9
16 Smple Lear Regresso Eample wegt versus temperature Temperature () Wegt (y) Wegt (y) Oservato temp wegt Et cetera Wegt (y) wegt temperature
17 Os. temp wegt -dff -dff. sq Et cetera mea SSX,8.5 SSY,9.8 SPXY6,75. SPy y SS : : NOTE: calculate frst, sce ts s eeded to calculate. From tese, te resduals (errors) for te equato, ad te sum of squared error (SSE) were calculated: Os. wegt y-pred resdual resdual sq Et cetera SSE: 5.89 Ad SSRSSY-SSE85.89 ANOVA Source df SS MS Model Error Total
18 F575.6 wt p. (very small) I ecel use: fdst(,df,df) to ota a p-value r :.97 Root MSE Or SE E :.57 BUT: Before terpretg te ANOVA tale, Are assumptos met? If assumptos were ot met, we would ave to make some trasformatos ad start over aga! resduals ( erro rs) Lear? resdual plot Equal varace? predcted wegt Idepedet oservatos? [eed aoter plot resduals versus tme or space, tat cause depedeces] 5 6
19 Normalty plot: Os. sorted Stad. Rel. Pro. resds resds Freq. z- dst Etc. cum ulatve proalty Questos: Proalty plot z-value. Are te assumptos of smple lear regresso met? Evdece? relatve frequecy Pro. z-dst.. If so, terpret f ts s a good equato ased o goodess of t measures.. Is te regresso sgfcat? 7 8
20 For 95% cofdece tervals for ad, would also eed estmated stadard errors: s s MSE + SS MSE SS Te t-value for 6 degrees of freedom ad te.975 percetle s. (tv(.5,6) EXCEL) ± t α, s For β o : 5.85 ±.. 75 t ± α, s For β :.568 ±.. 7 Est. Coeff St. Error For : For : CI: t(.975,6).. lower upper Questo: Could te real tercept e equal to? Gve a temperature of, wat s te estmated average wegt (predcted value) ad a 95% cofdece terval for ts estmate? 9 4
21 yˆ yˆ ( s s yˆ yˆ + ) MSE ( ) + SS ( 7.5) Gve a temperature of, wat s te estmated wegt for ay ew oservato, ad a 95% cofdece terval for ts estmate? yˆ yˆ ( + ) yˆ ± t, α syˆ s s yˆ yˆ MSE ( ) + SS ( 7.5) yˆ ± t, α syˆ
22 Multple Lear Regresso (MLR) For eample: Populato: y β + β + β +...+β p m +ε Sample: y p m +e yˆ K + m m e y β o s te y tercept parameter β, β, β,..., β m are slope parameters,,... m depedet varales ε - s te error term or resdual - s te varato te depedet varale (te y) wc s ot accouted for y te depedet varales (te s). For ay ftted equato (we ave te estmated parameters), we ca get te estmated average for te depedet varale, for ay set of s. Ts wll e te predcted value for y, wc s te estmated average of y, gve te partcular values for te varales. NOTE: I tet y Neter et al. pm+. Ts s ot e cofused wt te p- value dcatg sgfcace ypotess tests. yˆ Predcted log(vol) X log(d) +. X log(egt) were o -4.;. ;. estmated y fdg te least squared error soluto. Usg ts equato for d cm, egt8m, logte(d).48, logte(egt).45; logte(vol).5. volume (m ).84. Ts represets te estmated average volume for trees wt d cm ad egt8 m. Note: Ts equato s orgally a olear equato: vol a d Wc was trasformed to a lear equato usg logartms: log ( vol) log( a) + log( d) + c log( t) + logε Ad ts was ftted usg multple lear regresso t c ε 4 44
23 For te oservatos te sample data used to ft te regresso, we ca also get a estmate of te error (we ave measured volume). If te measured volume for ts tree was. m, or.477 log uts: error y yˆ For te ftted equato usg log uts. I orgal uts, te estmated error s NOTE: Ts s ot smply te atlog of -.6. Fdg te Set of Coeffcets tat Mmzes te Sum of Squared Errors Same process as for SLR: Fd te set of coeffcets tat results te mmum SSE, just tat tere are more parameters, terefore more partal dervatve equatos ad more equatos o E.g., wt -varales, tere wll e 4 coeffcets (tercept plus slopes) so four equatos For lear models, tere wll e oe uque matematcal soluto. For olear models, ts s ot possle ad we must searc to fd a soluto Usg te crtero of fdg te mamum lkelood (proalty) rater ta te mmum SSE, we would eed to searc for a soluto, eve for lear models (covered oter courses, e.g., FRST 5)
24 47 Least Squares Metod for MLR: Fd te set of estmated parameters (coeffcets) tat mmze sum of squared errors ( ) m p y e SSE )... ( m ) m( ) m( Take partal dervatves wt respect to eac of te coeffcets, set tem equal to zero ad solve. For tree -varales we ota: y SS SP SS SP SS y SP SS SP SS SP SS y SP SS SP SS SP SS y SP 48 Were SP dcates sum of products etwee two varales, for eample for y wt : ( )( ) ) ( s y y y y y SP y Ad SS dcates sums of squares for oe varale, for eample for : ( ) ) ( s SS
25 Propertes of a least squares regresso surface :. Always passes troug (,,,..., m, y). Sum of resduals s zero,.e., Σe. SSE te least possle (least squares) 4. Te slope for a partcular -varale s AFFECTED y correlato wt oter -varales: CANNOT terpret te slope for a partcular -varale, UNLESS t as zero correlato wt all oter - varales (or early zero f correlato s estmated from a sample). [class eample] Meetg Assumptos of MLR Oce coeffcets are otaed, we must ceck te assumptos of MLR efore we ca: assess goodess of ft (.e., ow well te regresso le fts te sample data) test sgfcace of te regresso calculate cofdece tervals ad test ypotess For tese test to e vald, assumptos of MLR cocerg te oservatos ad te errors (resduals) must e met. 49 5
26 Resdual Plots Assumptos of:. Te relatosp etwee te s ad y s lear VERY IMPORTANT!. Te varaces of te y values must e te same for every comato of te values.. Eac oservato (.e., s ad y ) must e depedet of all oter oservatos. ca e vsually cecked y usg RESIDUAL PLOTS Normalty Hstogram or Plot A fourt assumpto of te MLR s: 4. Te y values must e ormally dstruted for eac comato of values. A stogram of te errors, ad/or a ormalty plot ca e used to ceck ts, as well as tests of ormalty as wt SLR. Falure to meet tese assumptos wll result same prolems as wt SLR. A resdual plot sows te resdual (.e., y - ŷ ) as te y-as ad te predcted value ( ŷ ) as te -as. For te depece assumpto, te -as s tme or space tat eplas te depedece of te data. THIS IS THE SAME as for SLR. Look for prolems as wt SLR. Te effects of falg to meet a partcular assumpto are te same as for SLR Wat s dfferet? Sce tere are may varales, t wll e arder to decde wat to do to f ay prolems. 5 5
27 Eample: Lear relatosp met, equal varace, o evdece of tred wt oservato umer (depedece may e met). Also, ormal dstruto met. Logvolf(d,logd) R e s d u a l F requecy Normal Plot of Resduals Normal Score Resdual Model Dagostcs Hstogram of Resduals Resdual R e s d u a l R e s d u a l I Cart of Resduals Oservato Numer Resduals vs. Fts Ft X. 5 UCL.78 LCL Lear relatosp assumpto ot met Resdual Frequecy Normal Plot of Resduals Normal Score Hstogram of Resduals Resdual Volume versus d 4 Resdual Resdual I Cart of Resduals Oservato Numer 5 Ft Resduals vs. Fts 6 5 UCL.6 X. LCL-.6 54
28 Resdual Varaces are ot equal Frequecy Volume versus d squared ad d Normal Plot of Resduals - - Normal Score Hstogram of Resduals Resdual Resdual Resdual I Cart of Resduals Oservato Numer 5 Ft 5 5 Resduals vs. Fts X. 5 5 UCL.68 LCL Measuremets ad Samplg Assumptos Te remag assumptos of MLR are ased o te measuremets ad collecto of te samplg data, as wt SLR 5. Te values are measured wtout error (.e., te values are fed). 6. Te y values are radomly selected for eac gve set of te varales (.e., for eac fed set of values, a lst of all possle y values s made). As wt SLR, ofte oservatos wll e gatered usg smple radom samplg or systematc samplg (grd across te lad area). Ts does ot strctly meet ts assumpto [muc more dffcult to meet wt may - varales!] If te equato s correct, te ts does ot cause prolems. If ot, te estmated equato wll e ased. 56
29 Trasformatos Measures of Goodess of Ft Same as for SLR ecept tat tere are more varales; ca also add varales e.g. use d ad d as ad. Try to trasform s frst ad leave y varale of terest; ot always possle. Use graps to elp coose trasformatos Wll result a teratve process:. Ft te equato. Ceck te assumptos [ad ceck for outlers]. Make ay trasformatos ased o te resdual plot, ad plots of y versus eac 4. Also, ceck ay very uusual pots to see f tese are measuremet/trascrpto errors; ONLY remove te oservato f tere s a very good reaso to do so 5. Ft te equato aga, ad ceck te assumptos 6. Cotue utl te assumptos are met [or early met] How well does te regresso ft te sample data? For multple lear regresso, a grap of te te predcted versus measured y values dcates ow well te le fts te data Two measures commoly used: coeffcet of multple determato (R ) ad stadard error of te estmate(se E ), smlar to SLR To calculate R ad SE E, frst, calculate te SSE (ts s wat was mmzed, as wt SLR): SSE ( y yˆ ) ( y ( m m )) e Te sum of squared dffereces etwee te measured ad estmated y s. Ts s te same as for SLR, ut tere are more slopes ad more (predctor) varales
30 Calculate te sum of squares for y: ( ) SSy y y y y sy ( ) Te sum of squared dfferece etwee te measured y ad te mea of y-measures. Calculate te sum of squares regresso: SSreg ( y yˆ ) SSy SSE SP y + SP y SP y Te sum of squared dffereces etwee te mea of y- measures ad te predcted y s from te ftted equato. Also, s te sum of squares for y te sum of squared errors. SSy SSE SSE SSreg Te: R SSy SSy SSy SSE, SSY are ased o y s used te equato wll ot e orgal uts f y was trasformed R coeffcet of multple determato; proporto of varace of y, accouted for y te regresso usg s O (very poor orzotal surface represetg o relatosp etwee y ad s) to (perfect ft surface passes troug te data) SSE falls as m (umer of depedet varale) creases, so R rses as more eplaatory (depedet or predctor) varales are added. A smlar measure s called te Adjusted R value. A pealty s added as you add -varales to te equato: R a SSE m ( + ) SSy 59 6
31 SSE Ad: SE E m SSE s ased o y s used te equato wll ot e orgal uts f y was trasformed -m- s te degrees of freedom for te error; s te umer of oservatos mus te umer of ftted coeffcets SE E - stadard error of te estmate; same uts as y Uder ormalty of te errors: o ± SE E 68% of sample oservatos o ± SE E 95% of sample oservatos Wat low SE E SE E falls as te umer of predctor varales creases ad SSE falls, ut te rses, sce -m - s gettg smaller y-varale was trasformed: Ca calculate estmates of tese for te orgal y-varale ut, I (Ft Ide) ad estmated stadard error of te estmate (SE E ), order to compare to R ad SE E of oter equatos were te y was ot trasformed, smlar to SLR. I - SSE/SSY were SSE, SSY are orgal uts. NOTE must ack-trasform te predcted y s to calculate te SSE orgal uts. Does ot ave te same propertes as R, owever t ca e less ta Estmated stadard error of te estmate (SE E ), we te depedet varale, y, as ee trasformed: SSE( orgal uts) SE E ' m SEE - stadard error of te estmate ; same uts as orgal uts for te depedet varale wat low SEE 6 6
32 Estmated Varaces, Cofdece Itervals ad Hypotess Tests Testg Weter te Regresso s Sgfcat Does kowledge of s mprove te estmate of te mea of y? Or s t a flat surface, wc meas we sould just use te mea of y as a estmate of mea y for ay set of values? SSE/ (-m-): Mea squared error. o Te degrees of freedom are -m- (same as -(m+) o oservatos wt (m+) statstcs estmated from tese:,,, m Uder te assumptos of MLR, s a uased estmated of te true varace of te error terms (error varace) SSR/m: Called te Mea Square Regresso Degrees of Freedomm: m -varales Uder te assumptos of MLR, ts s a estmate te error varace PLUS a term of varace eplaed y te regresso usg s. H: Regresso s ot sgfcat H: Regresso s sgfcat Same as: H: β β β... β m [all slopes are zero meag o relatosp wt s] H: ot all slopes [some or all slopes are ot equal to zero] If H s true, te te equato s: y β m +ε y β ˆ + ε y β Were te -varales ave o fluece over y; tey do ot elp to etter estmate y. 6 64
33 As wt SLR, we ca use a F-test, as t s te rato of two varaces; ulke SLR we caot use a t-test sce we are oly testg several slope coeffcets. Usg a F test statstc: SSreg m F SSE ( m ) MSreg MSE Uder H, ts follows a F dstruto for a - α percetle wt m ad -m- degrees of freedom. If te F for te ftted equato s larger ta te F from te tale, we reject H (ot lkely true). Te regresso s sgfcat, tat oe or more of te te true slopes (te populato slopes) are lkely ot equal to zero. Iformato for te F-test te Aalyss of Varace Tale: Source df SS MS F p-value Regresso m MSreg F Pro F> SSreg SSreg/m MSreg/MSE F (m,-m-,- α) Error -m- SSE MSE SSE/( m-) Estmated Stadard Errors for te Slope ad Itercept Uder te assumptos, we ca ota a uased estmated of te stadard errors for te slope ad for te tercept [measure of ow tese would vary amog dfferet sample sets], usg te oe set of sample data. For multple lear regresso, tese are more easly calculated usg matr algera. If tere are more ta - varales, te calculatos ecome dffcult; we wll rely o statstcal packages to do tese calculatos. Cofdece Itervals for te True Slope ad Itercept Uder te assumptos, cofdece tervals ca e calculated as: For β o : ± t α, m s For β j : j ± t α, m s [ for ay of te slopes] j Total - SSy [See eample] [See eample] 65 66
34 Hypotess Tests for oe of te True Slopes or Itercept H: β j c [te parameter (true tercept or true slope s equal to te costat, c, gve tat te oter -varales are te equato] H: β j c [true tercept or slope dffers from te costat c; gve tat te oter -varales are te equato] Te regresso s sgfcat, ut wc -varales sould we reta? Wt MLR, we are partcularly terested wc - varales to reta. We te test: Is varale j sgfcat gve te oter varales? e.g. dameter, egt - do we eed ot? Test statstc: j c t s Uder H, ts s dstruted as a t value of t c t -m-, -α/. Reject H o f t > t c. It s possle to do oe-sded ypoteses also, were te alteratve s tat te true parameter (slope or tercept) s greater ta (or less ta) a specfed costat c. MUST e careful wt te t c as ts s dfferet. [See eample] j H: β j, gve oter -varales (.e., varale ot sgfcat) H: β j, gve oter -varales. A t-test for tat varale ca e used to test ts
35 Aoter test, te partal F-test ca e used to test oe - varale (as t-test) or to test a group of -varales, gve te oter -varales te equato. Get regresso aalyss results for all -varales [full model] Get regresso aalyss results for all ut te -varales to e tested [reduced model] partal F OR partal F ( SSreg( full) SSreg( reduced) ) SSE ( m )( full) ( SSE( reduced) SSE( full) ) SSE ( m )( full) ( SS due to dropped varale(s)) /r MSE( full) Were r s te umer of -varales tat were dropped (also equals: ()te regresso degrees of freedom for te full model mus te regresso degrees of freedom for te reduced model, OR () te error degrees of freedom for te reduced model, mus te error degrees of freedom for te full model) r r Uder H, ts follows a F dstruto for a - α percetle wt r ad -m- (full model) degrees of freedom. If te F for te ftted equato s larger ta te F from te tale, we reject H (ot lkely true). Te regresso s sgfcat, tat te varale(s) tat were dropped are sgfcat (accout for varace of te y-varale), gve tat te oter -varales are te model. [See eample wt te use of class varales, ut ca e for ay suset of -varales] 69 7
36 Cofdece Iterval for te True Mea of y gve a partcular set of values For te mea of all possle y-values gve a partcular value set of -values (μ y ): were yˆ yˆ ± t m, α syˆ L+ m m yˆ s yˆ ( ew) L+ m from statstcal package output For te average of g ew possle y-values gve a partcular value of : were yˆ yˆ ( ew) ± t m, α syˆ ( ewg ) + + m + L+ m m s yˆ from statstcal package output Cofdece Bads Plot of te cofdece tervals for te mea of y for several sets -values s ot possle wt MLR s yˆ ( ewg) [See eample] from statstcal package output Predcto Iterval for or more y-values gve a partcular set of values For oe possle ew y-value gve a partcular set of values: Were yˆ ( ew) ± t m, α syˆ ( ew) 7 7
37 Selectg ad Comparg Alteratve Models Process to Ft a Equato usg Least Squares Steps (same as for SLR):. Sample data are eeded, o wc te depedet varale ad all eplaatory (depedet) varales are measured.. Make ay trasformatos tat are eeded to meet te most crtcal assumpto: Te relatosp etwee y ad s s lear. Eample: volume β + β d +β d may e lear wereas volume versus d s ot. Need ot varales.. Ft te equato to mmze te sum of squared error. 4. Ceck Assumptos. If ot met, go ack to Step. 5. If assumptos are met, te ceck f te regresso s sgfcat. If t s ot, te t s ot a caddate model (eed oter -varales). If yes, te go troug furter steps for MLR. 6. Are all varales eeded? If tere are -varales tat are ot sgfcat, gve te oter varales: drop te least sgfcat oe (gest p-value, or lowest asolute value of t) reft te regresso ad ceck assumptos. f assumptos are met, te repeat steps 5 ad 6 cotue utl all varales te regresso are sgfcat gve te oter -varales also te model 7 74
38 Metods to ad selectg predctor () varales Metods ave ee developed to elp coosg wc - varales to clude te equato. Tese clude:. Forward: Brg varales oe at a tme, utl te remag oes are o loger sgfcat, gve te oters already te equato. ( oly). Backward: Drop varales oe at a tme, utl all remag varales are sgfcat, gve te oters stll te equato (out oly). Stepwse ( ad out) NOTE: Tese tools just gves caddate models. You must ceck weter te assumptos are met ad do a full assessmet of te regresso results Steps for Forward Stepwse, for eample: To ft ts y ad, you would eed to do te followg steps:. Ft a smple lear regresso for vol/a wt eac of te eplaatory () varales.. Of te equatos tat are sgfcat (assumptos met?), select te oe wt te gest F-value.. Ft a MLR wt vol/a usg te selected varale, plus eac of te eplaatory varales ( -varales eac equatos). Ceck to see f te ew varale s sgfcat gve te orgal varale (wc may ow e ot sgfcat, ut forward stepwse does ot drop varales). Of te oes tat are sgfcat (gve te orgal varale s also te equato), pck te oe wt te largest partal-f (for te ew varale). 4. Repeat step, rgg varales utl ) tere are o more varales or ) te remag varales are ot sgfcat gve te oter varales
39 SAS Outputs: forward stepwse Te REG Procedure Numer of Oservatos Read 8 Numer of Oservatos Used 8 Te REG Procedure Model: MODEL Depedet Varale: vola vola Numer of Oservatos Read 8 Numer of Oservatos Used 8 Forward Selecto: Step Descrptve Statstcs Ucorrected Varale Sum Mea SS Varace Itercept aa stemsa qd age s topt vola Descrptve Statstcs Stadard Varale Devato Lael Itercept Itercept aa aa stemsa stemsa qd qd age.855 age s.7684 s topt.9448 topt vola vola Varale aa Etered: R-Square.77 ad C(p) 87.5 Aalyss of Varace Sum of Mea Source DF Squares Square F Value Pr > F Model <. Error Corrected Total Parameter Stadard Type II F Varale Estmate Error SS Value Pr>F Itercept aa <. Bouds o codto umer:, 77 78
40 Forward Selecto: Step Varale topt Etered: R-Square.985 ad C(p) Aalyss of Varace Sum of Mea Source DF Squares Square F Value Pr > F Model <. Error Corrected Total Parameter Stadard Type F Varale Estmate Error II SS Value Pr> F Itercept <. aa <. topt <. Bouds o codto umer:.5, 4.5 Forward Selecto: Step Varale stemsa Etered: R-Square.9879 ad C(p).6949 Aalyss of Varace Sum of Mea Source DF Squares Square F Value Pr > F Model <. Error Corrected Total Parameter Stadard Type Varale Estmate Error II SS F Value Pr> F Itercept <. aa <. stemsa topt <. Bouds o codto umer: 4.4, No oter varale met te.5 sgfcace level for etry to te model. 79 8
41 Summary of Forward Selecto Step Numer Partal Model Vars I R-Square R-Square C(p) F Value aa topt stemsa Summary of Forward Selecto Step Pr > F <. <..94 For a umer of models, select ased o:. Meetg assumptos: If a equato does ot meet te assumpto of a lear relatosp, t s ot a caddate model. Compare te ft statstcs. Select ger R (or I ), ad lower SE E (or SE E ). Reject ay models were te regresso s ot sgfcat, sce ts model s o etter ta just usg te mea of y as te predcted value. 4. Select a model tat s ologcally tractale. A smpler model s geerally preferred, uless tere are practcal/ologcal reasos to select te more comple model 5. Cosder te cost of usg te model 8 8
42 Addg class varales as predctors Wat to add a class varale. Eamples:. Add speces to a equato to estmate tree egt.. Add geder (male/female) to a equato to estmate wegt of adult taled frogs.. Add mace type to a equato tat predcts lumer output. How s ts doe? Use dummy or dcator varales to represet te class varale e.g. ave speces. Set up X ad X as dummy varales: Speces X X Cedar Hemlock Douglas fr Oly eed two dummy varales to represet te tree speces. Te two dummy varales as a group represet te speces. Add te dummy varales to te equato ts wll alter te tercept To alter te slopes, add a teracto etwee dummy varales ad cotuous varale(s) e.g. ave speces, ad a cotuous varale, d Speces X X d X4X * d X5X*d Cedar Hemlock Douglas fr 5 NOTE: Tere would e more ta oe le of data (sample) for eac speces. o Te two dummy varales, ad te teractos wt te cotuous varale as a group represet te speces. 8 84
43 How does ts work? y { e For Cedar (CW): dummy varales d teractos Oter metods, ta SLR ad Multple Lear Regresso, we trasformatos do ot work: Nolear least squares: Least squares soluto for olear models; uses a searc algortm to fd estmated coeffcets; as good propertes for large datasets; stll assumes ormalty, equal varaces, ad depedet oservatos For Hemlock (HW): Wegted least squares: for uequal varaces. Estmate te varaces ad use tese wegtg te least squares ft of te regresso; assumes ormalty ad depedet oservatos For Douglas fr (FD): Geearlzed lear model: used for dstrutos oter ta ormal (e.g., omal, Posso, etc.), ut wt o correlato etwee oservatos; uses mamum lkelood Terefore: ft oe equato usg all data, ut get dfferet equatos for dfferet speces. Also, ca test for dffereces amog speces, usg a partal-f test. Geeralzed least Squares ad Med Models: use mamum lkelood for fttg models wt uequal varaces, correlatos over space, correlatos over tme, ut ormally dstruted errors Geeralzed lear med models: Allows for uequal varaces, correlatos over space ad/or tme, ad o-ormal dstrutos; uses mamum lkelood 85 86
Simple Linear Regression
Statstcal Methods I (EST 75) Page 139 Smple Lear Regresso Smple regresso applcatos are used to ft a model descrbg a lear relatoshp betwee two varables. The aspects of least squares regresso ad correlato
More informationSTA302/1001-Fall 2008 Midterm Test October 21, 2008
STA3/-Fall 8 Mdterm Test October, 8 Last Name: Frst Name: Studet Number: Erolled (Crcle oe) STA3 STA INSTRUCTIONS Tme allowed: hour 45 mutes Ads allowed: A o-programmable calculator A table of values from
More informationObjectives of Multiple Regression
Obectves of Multple Regresso Establsh the lear equato that best predcts values of a depedet varable Y usg more tha oe eplaator varable from a large set of potetal predctors {,,... k }. Fd that subset of
More informationresidual. (Note that usually in descriptions of regression analysis, upper-case
Regresso Aalyss Regresso aalyss fts or derves a model that descres the varato of a respose (or depedet ) varale as a fucto of oe or more predctor (or depedet ) varales. The geeral regresso model s oe of
More informationMultiple Regression. More than 2 variables! Grade on Final. Multiple Regression 11/21/2012. Exam 2 Grades. Exam 2 Re-grades
STAT 101 Dr. Kar Lock Morga 11/20/12 Exam 2 Grades Multple Regresso SECTIONS 9.2, 10.1, 10.2 Multple explaatory varables (10.1) Parttog varablty R 2, ANOVA (9.2) Codtos resdual plot (10.2) Trasformatos
More informationExample: Multiple linear regression. Least squares regression. Repetition: Simple linear regression. Tron Anders Moger
Example: Multple lear regresso 5000,00 4000,00 Tro Aders Moger 0.0.007 brthweght 3000,00 000,00 000,00 0,00 50,00 00,00 50,00 00,00 50,00 weght pouds Repetto: Smple lear regresso We defe a model Y = β0
More informationLecture 7. Confidence Intervals and Hypothesis Tests in the Simple CLR Model
Lecture 7. Cofdece Itervals ad Hypothess Tests the Smple CLR Model I lecture 6 we troduced the Classcal Lear Regresso (CLR) model that s the radom expermet of whch the data Y,,, K, are the outcomes. The
More informationStatistics MINITAB - Lab 5
Statstcs 10010 MINITAB - Lab 5 PART I: The Correlato Coeffcet Qute ofte statstcs we are preseted wth data that suggests that a lear relatoshp exsts betwee two varables. For example the plot below s of
More informationChapter 13 Student Lecture Notes 13-1
Chapter 3 Studet Lecture Notes 3- Basc Busess Statstcs (9 th Edto) Chapter 3 Smple Lear Regresso 4 Pretce-Hall, Ic. Chap 3- Chapter Topcs Types of Regresso Models Determg the Smple Lear Regresso Equato
More informationESS Line Fitting
ESS 5 014 17. Le Fttg A very commo problem data aalyss s lookg for relatoshpetwee dfferet parameters ad fttg les or surfaces to data. The smplest example s fttg a straght le ad we wll dscuss that here
More informationLecture Notes Types of economic variables
Lecture Notes 3 1. Types of ecoomc varables () Cotuous varable takes o a cotuum the sample space, such as all pots o a le or all real umbers Example: GDP, Polluto cocetrato, etc. () Dscrete varables fte
More informationENGI 3423 Simple Linear Regression Page 12-01
ENGI 343 mple Lear Regresso Page - mple Lear Regresso ometmes a expermet s set up where the expermeter has cotrol over the values of oe or more varables X ad measures the resultg values of aother varable
More informationChapter Two. An Introduction to Regression ( )
ubject: A Itroducto to Regresso Frst tage Chapter Two A Itroducto to Regresso (018-019) 1 pg. ubject: A Itroducto to Regresso Frst tage A Itroducto to Regresso Regresso aalss s a statstcal tool for the
More informationRegresso What s a Model? 1. Ofte Descrbe Relatoshp betwee Varables 2. Types - Determstc Models (o radomess) - Probablstc Models (wth radomess) EPI 809/Sprg 2008 9 Determstc Models 1. Hypothesze
More informationStatistics: Unlocking the Power of Data Lock 5
STAT 0 Dr. Kar Lock Morga Exam 2 Grades: I- Class Multple Regresso SECTIONS 9.2, 0., 0.2 Multple explaatory varables (0.) Parttog varablty R 2, ANOVA (9.2) Codtos resdual plot (0.2) Exam 2 Re- grades Re-
More informationProbability and. Lecture 13: and Correlation
933 Probablty ad Statstcs for Software ad Kowledge Egeers Lecture 3: Smple Lear Regresso ad Correlato Mocha Soptkamo, Ph.D. Outle The Smple Lear Regresso Model (.) Fttg the Regresso Le (.) The Aalyss of
More information: At least two means differ SST
Formula Card for Eam 3 STA33 ANOVA F-Test: Completely Radomzed Desg ( total umber of observatos, k = Number of treatmets,& T = total for treatmet ) Step : Epress the Clam Step : The ypotheses: :... 0 A
More informationLinear Regression Siana Halim
Lear Regresso Saa Halm Draper,N.R; Smth, H.; Appled Regresso Aalyss,3rd Edto, Joh Wley & Sos, Ic. 998 Motgomery, D.C; Peck, E.A; Itroducto to Lear Regresso Aalyss, d Edto, 99 Outle Itroducto Fttg a Straght
More informationLinear Regression with One Regressor
Lear Regresso wth Oe Regressor AIM QA.7. Expla how regresso aalyss ecoometrcs measures the relatoshp betwee depedet ad depedet varables. A regresso aalyss has the goal of measurg how chages oe varable,
More information12.2 Estimating Model parameters Assumptions: ox and y are related according to the simple linear regression model
1. Estmatg Model parameters Assumptos: ox ad y are related accordg to the smple lear regresso model (The lear regresso model s the model that says that x ad y are related a lear fasho, but the observed
More informationMultiple Linear Regression Analysis
LINEA EGESSION ANALYSIS MODULE III Lecture - 4 Multple Lear egresso Aalyss Dr. Shalabh Departmet of Mathematcs ad Statstcs Ida Isttute of Techology Kapur Cofdece terval estmato The cofdece tervals multple
More informationSummary of the lecture in Biostatistics
Summary of the lecture Bostatstcs Probablty Desty Fucto For a cotuos radom varable, a probablty desty fucto s a fucto such that: 0 dx a b) b a dx A probablty desty fucto provdes a smple descrpto of the
More informationECON 482 / WH Hong The Simple Regression Model 1. Definition of the Simple Regression Model
ECON 48 / WH Hog The Smple Regresso Model. Defto of the Smple Regresso Model Smple Regresso Model Expla varable y terms of varable x y = β + β x+ u y : depedet varable, explaed varable, respose varable,
More informationb. There appears to be a positive relationship between X and Y; that is, as X increases, so does Y.
.46. a. The frst varable (X) s the frst umber the par ad s plotted o the horzotal axs, whle the secod varable (Y) s the secod umber the par ad s plotted o the vertcal axs. The scatterplot s show the fgure
More informationSimple Linear Regression
Correlato ad Smple Lear Regresso Berl Che Departmet of Computer Scece & Iformato Egeerg Natoal Tawa Normal Uversty Referece:. W. Navd. Statstcs for Egeerg ad Scetsts. Chapter 7 (7.-7.3) & Teachg Materal
More informationSTA 108 Applied Linear Models: Regression Analysis Spring Solution for Homework #1
STA 08 Appled Lear Models: Regresso Aalyss Sprg 0 Soluto for Homework #. Let Y the dollar cost per year, X the umber of vsts per year. The the mathematcal relato betwee X ad Y s: Y 300 + X. Ths s a fuctoal
More informationLecture 8: Linear Regression
Lecture 8: Lear egresso May 4, GENOME 56, Sprg Goals Develop basc cocepts of lear regresso from a probablstc framework Estmatg parameters ad hypothess testg wth lear models Lear regresso Su I Lee, CSE
More informationPrevious lecture. Lecture 8. Learning outcomes of this lecture. Today. Statistical test and Scales of measurement. Correlation
Lecture 8 Emprcal Research Methods I434 Quattatve Data aalss II Relatos Prevous lecture Idea behd hpothess testg Is the dfferece betwee two samples a reflecto of the dfferece of two dfferet populatos or
More informationEconometric Methods. Review of Estimation
Ecoometrc Methods Revew of Estmato Estmatg the populato mea Radom samplg Pot ad terval estmators Lear estmators Ubased estmators Lear Ubased Estmators (LUEs) Effcecy (mmum varace) ad Best Lear Ubased Estmators
More informationChapter 8. Inferences about More Than Two Population Central Values
Chapter 8. Ifereces about More Tha Two Populato Cetral Values Case tudy: Effect of Tmg of the Treatmet of Port-We tas wth Lasers ) To vestgate whether treatmet at a youg age would yeld better results tha
More informationChapter 14 Logistic Regression Models
Chapter 4 Logstc Regresso Models I the lear regresso model X β + ε, there are two types of varables explaatory varables X, X,, X k ad study varable y These varables ca be measured o a cotuous scale as
More informationCorrelation and Simple Linear Regression
Correlato ad Smple Lear Regresso Berl Che Departmet of Computer Scece & Iformato Egeerg Natoal Tawa Normal Uverst Referece:. W. Navd. Statstcs for Egeerg ad Scetsts. Chapter 7 (7.-7.3) & Teachg Materal
More information4. Standard Regression Model and Spatial Dependence Tests
4. Stadard Regresso Model ad Spatal Depedece Tests Stadard regresso aalss fals the presece of spatal effects. I case of spatal depedeces ad/or spatal heterogeet a stadard regresso model wll be msspecfed.
More informationChapter Business Statistics: A First Course Fifth Edition. Learning Objectives. Correlation vs. Regression. In this chapter, you learn:
Chapter 3 3- Busess Statstcs: A Frst Course Ffth Edto Chapter 2 Correlato ad Smple Lear Regresso Busess Statstcs: A Frst Course, 5e 29 Pretce-Hall, Ic. Chap 2- Learg Objectves I ths chapter, you lear:
More informationSimple Linear Regression and Correlation.
Smple Lear Regresso ad Correlato. Correspods to Chapter 0 Tamhae ad Dulop Sldes prepared b Elzabeth Newto (MIT) wth some sldes b Jacquele Telford (Johs Hopks Uverst) Smple lear regresso aalss estmates
More informationIdea is to sample from a different distribution that picks points in important regions of the sample space. Want ( ) ( ) ( ) E f X = f x g x dx
Importace Samplg Used for a umber of purposes: Varace reducto Allows for dffcult dstrbutos to be sampled from. Sestvty aalyss Reusg samples to reduce computatoal burde. Idea s to sample from a dfferet
More informationMultiple Choice Test. Chapter Adequacy of Models for Regression
Multple Choce Test Chapter 06.0 Adequac of Models for Regresso. For a lear regresso model to be cosdered adequate, the percetage of scaled resduals that eed to be the rage [-,] s greater tha or equal to
More informationREVIEW OF SIMPLE LINEAR REGRESSION SIMPLE LINEAR REGRESSION
REVIEW OF SIMPLE LINEAR REGRESSION SIMPLE LINEAR REGRESSION I lear regreo, we coder the frequecy dtrbuto of oe varable (Y) at each of everal level of a ecod varable (X). Y kow a the depedet varable. The
More informationMidterm Exam 1, section 1 (Solution) Thursday, February hour, 15 minutes
coometrcs, CON Sa Fracsco State Uversty Mchael Bar Sprg 5 Mdterm am, secto Soluto Thursday, February 6 hour, 5 mutes Name: Istructos. Ths s closed book, closed otes eam.. No calculators of ay kd are allowed..
More informationLecture 3. Sampling, sampling distributions, and parameter estimation
Lecture 3 Samplg, samplg dstrbutos, ad parameter estmato Samplg Defto Populato s defed as the collecto of all the possble observatos of terest. The collecto of observatos we take from the populato s called
More informationMean is only appropriate for interval or ratio scales, not ordinal or nominal.
Mea Same as ordary average Sum all the data values ad dvde by the sample sze. x = ( x + x +... + x Usg summato otato, we wrte ths as x = x = x = = ) x Mea s oly approprate for terval or rato scales, ot
More informationStatistics. Correlational. Dr. Ayman Eldeib. Simple Linear Regression and Correlation. SBE 304: Linear Regression & Correlation 1/3/2018
/3/08 Sstems & Bomedcal Egeerg Departmet SBE 304: Bo-Statstcs Smple Lear Regresso ad Correlato Dr. Ama Eldeb Fall 07 Descrptve Orgasg, summarsg & descrbg data Statstcs Correlatoal Relatoshps Iferetal Geeralsg
More informationApplied Statistics and Probability for Engineers, 5 th edition February 23, b) y ˆ = (85) =
Appled Statstcs ad Probablty for Egeers, 5 th edto February 3, y.8.7.6.5.4.3.. -5 5 5 x b) y ˆ.3999 +.46(85).6836 c) y ˆ.3999 +.46(9).744 d) ˆ.46-3 a) Regresso Aalyss: Ratg Pots versus Meters per Att The
More informationOrdinary Least Squares Regression. Simple Regression. Algebra and Assumptions.
Ordary Least Squares egresso. Smple egresso. Algebra ad Assumptos. I ths part of the course we are gog to study a techque for aalysg the lear relatoshp betwee two varables Y ad X. We have pars of observatos
More informationTHE ROYAL STATISTICAL SOCIETY HIGHER CERTIFICATE
THE ROYAL STATISTICAL SOCIETY 00 EXAMINATIONS SOLUTIONS HIGHER CERTIFICATE PAPER I STATISTICAL THEORY The Socety provdes these solutos to assst caddates preparg for the examatos future years ad for the
More informationChapter 13, Part A Analysis of Variance and Experimental Design. Introduction to Analysis of Variance. Introduction to Analysis of Variance
Chapter, Part A Aalyss of Varace ad Epermetal Desg Itroducto to Aalyss of Varace Aalyss of Varace: Testg for the Equalty of Populato Meas Multple Comparso Procedures Itroducto to Aalyss of Varace Aalyss
More informationSimple Linear Regression - Scalar Form
Smple Lear Regresso - Scalar Form Q.. Model Y X,..., p..a. Derve the ormal equatos that mmze Q. p..b. Solve for the ordary least squares estmators, p..c. Derve E, V, E, V, COV, p..d. Derve the mea ad varace
More informationGeneral Method for Calculating Chemical Equilibrium Composition
AE 6766/Setzma Sprg 004 Geeral Metod for Calculatg Cemcal Equlbrum Composto For gve tal codtos (e.g., for gve reactats, coose te speces to be cluded te products. As a example, for combusto of ydroge wt
More informationThe equation is sometimes presented in form Y = a + b x. This is reasonable, but it s not the notation we use.
INTRODUCTORY NOTE ON LINEAR REGREION We have data of the form (x y ) (x y ) (x y ) These wll most ofte be preseted to us as two colum of a spreadsheet As the topc develops we wll see both upper case ad
More informationWu-Hausman Test: But if X and ε are independent, βˆ. ECON 324 Page 1
Wu-Hausma Test: Detectg Falure of E( ε X ) Caot drectly test ths assumpto because lack ubased estmator of ε ad the OLS resduals wll be orthogoal to X, by costructo as ca be see from the momet codto X'
More informationUNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS
UNIVERSITY OF OSLO DEPARTMENT OF ECONOMICS Exam: ECON430 Statstcs Date of exam: Frday, December 8, 07 Grades are gve: Jauary 4, 08 Tme for exam: 0900 am 00 oo The problem set covers 5 pages Resources allowed:
More informationLecture 1 Review of Fundamental Statistical Concepts
Lecture Revew of Fudametal Statstcal Cocepts Measures of Cetral Tedecy ad Dsperso A word about otato for ths class: Idvduals a populato are desgated, where the dex rages from to N, ad N s the total umber
More informationbest estimate (mean) for X uncertainty or error in the measurement (systematic, random or statistical) best
Error Aalyss Preamble Wheever a measuremet s made, the result followg from that measuremet s always subject to ucertaty The ucertaty ca be reduced by makg several measuremets of the same quatty or by mprovg
More informationCLASS NOTES. for. PBAF 528: Quantitative Methods II SPRING Instructor: Jean Swanson. Daniel J. Evans School of Public Affairs
CLASS NOTES for PBAF 58: Quattatve Methods II SPRING 005 Istructor: Jea Swaso Dael J. Evas School of Publc Affars Uversty of Washgto Ackowledgemet: The structor wshes to thak Rachel Klet, Assstat Professor,
More informationTESTS BASED ON MAXIMUM LIKELIHOOD
ESE 5 Toy E. Smth. The Basc Example. TESTS BASED ON MAXIMUM LIKELIHOOD To llustrate the propertes of maxmum lkelhood estmates ad tests, we cosder the smplest possble case of estmatg the mea of the ormal
More informationRecall MLR 5 Homskedasticity error u has the same variance given any values of the explanatory variables Var(u x1,...,xk) = 2 or E(UU ) = 2 I
Chapter 8 Heterosedastcty Recall MLR 5 Homsedastcty error u has the same varace gve ay values of the eplaatory varables Varu,..., = or EUU = I Suppose other GM assumptos hold but have heterosedastcty.
More informationAnalysis of Variance with Weibull Data
Aalyss of Varace wth Webull Data Lahaa Watthaacheewaul Abstract I statstcal data aalyss by aalyss of varace, the usual basc assumptos are that the model s addtve ad the errors are radomly, depedetly, ad
More informationHomework Solution (#5)
Homework Soluto (# Chapter : #6,, 8(b, 3, 4, 44, 49, 3, 9 ad 7 Chapter. Smple Lear Regresso ad Correlato.6 (6 th edto 7, old edto Page 9 Rafall volume ( vs Ruoff volume ( : 9 8 7 6 4 3 : a. Yes, the scatter-plot
More informationECONOMETRIC THEORY. MODULE VIII Lecture - 26 Heteroskedasticity
ECONOMETRIC THEORY MODULE VIII Lecture - 6 Heteroskedastcty Dr. Shalabh Departmet of Mathematcs ad Statstcs Ida Isttute of Techology Kapur . Breusch Paga test Ths test ca be appled whe the replcated data
More informationUNIVERSITY OF TORONTO AT SCARBOROUGH. Sample Exam STAC67. Duration - 3 hours
UNIVERSITY OF TORONTO AT SCARBOROUGH Sample Exam STAC67 Durato - 3 hours AIDS ALLOWED: THIS EXAM IS OPEN BOOK (NOTES) Calculator (No phoe calculators are allowed) LAST NAME FIRST NAME STUDENT NUMBER There
More informationSum Mean n
tatstcal Methods I (EXT 75) Page 147 ummary data Itermedate Calculatos X = 83 Y = 8 X = 51 Y = 368 Mea of X = X = 5.1875 Mea of Y = Y = 14.5 XY = 1348 = 16 Correcto factors ad Corrected values (ums of
More informationLecture 2: Linear Least Squares Regression
Lecture : Lear Least Squares Regresso Dave Armstrog UW Mlwaukee February 8, 016 Is the Relatoshp Lear? lbrary(car) data(davs) d 150) Davs$weght[d]
More informationFundamentals of Regression Analysis
Fdametals of Regresso Aalyss Regresso aalyss s cocered wth the stdy of the depedece of oe varable, the depedet varable, o oe or more other varables, the explaatory varables, wth a vew of estmatg ad/or
More informationLecture 2: The Simple Regression Model
Lectre Notes o Advaced coometrcs Lectre : The Smple Regresso Model Takash Yamao Fall Semester 5 I ths lectre we revew the smple bvarate lear regresso model. We focs o statstcal assmptos to obta based estmators.
More informationChapter 11 The Analysis of Variance
Chapter The Aalyss of Varace. Oe Factor Aalyss of Varace. Radomzed Bloc Desgs (ot for ths course) NIPRL . Oe Factor Aalyss of Varace.. Oe Factor Layouts (/4) Suppose that a expermeter s terested populatos
More informationMidterm Exam 1, section 2 (Solution) Thursday, February hour, 15 minutes
coometrcs, CON Sa Fracsco State Uverst Mchael Bar Sprg 5 Mdterm xam, secto Soluto Thursda, Februar 6 hour, 5 mutes Name: Istructos. Ths s closed book, closed otes exam.. No calculators of a kd are allowed..
More informationJohns Hopkins University Department of Biostatistics Math Review for Introductory Courses
Johs Hopks Uverst Departmet of Bostatstcs Math Revew for Itroductor Courses Ratoale Bostatstcs courses wll rel o some fudametal mathematcal relatoshps, fuctos ad otato. The purpose of ths Math Revew s
More informationExample. Row Hydrogen Carbon
SMAM 39 Least Squares Example. Heatg ad combusto aalyses were performed order to study the composto of moo rocks collected by Apollo 4 ad 5 crews. Recorded c ad c of the Mtab output are the determatos
More informationε. Therefore, the estimate
Suggested Aswers, Problem Set 3 ECON 333 Da Hugerma. Ths s ot a very good dea. We kow from the secod FOC problem b) that ( ) SSE / = y x x = ( ) Whch ca be reduced to read y x x = ε x = ( ) The OLS model
More informationInvestigation of Partially Conditional RP Model with Response Error. Ed Stanek
Partally Codtoal Radom Permutato Model 7- vestgato of Partally Codtoal RP Model wth Respose Error TRODUCTO Ed Staek We explore the predctor that wll result a smple radom sample wth respose error whe a
More informationC. Statistics. X = n geometric the n th root of the product of numerical data ln X GM = or ln GM = X 2. X n X 1
C. Statstcs a. Descrbe the stages the desg of a clcal tral, takg to accout the: research questos ad hypothess, lterature revew, statstcal advce, choce of study protocol, ethcal ssues, data collecto ad
More informationLinear Regression. Can height information be used to predict weight of an individual? How long should you wait till next eruption?
Iter-erupto Tme Weght Correlato & Regreo 1 1 Lear Regreo 0 80 70 80 Heght 1 Ca heght formato be ued to predct weght of a dvdual? How log hould ou wat tll et erupto? Weght: Repoe varable (Outcome, Depedet)
More informationConvergence of the Desroziers scheme and its relation to the lag innovation diagnostic
Covergece of the Desrozers scheme ad ts relato to the lag ovato dagostc chard Méard Evromet Caada, Ar Qualty esearch Dvso World Weather Ope Scece Coferece Motreal, August 9, 04 o t t O x x x y x y Oservato
More informationSTATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS. x, where. = y - ˆ " 1
STATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS Recall Assumpto E(Y x) η 0 + η x (lear codtoal mea fucto) Data (x, y ), (x 2, y 2 ),, (x, y ) Least squares estmator ˆ E (Y x) ˆ " 0 + ˆ " x, where ˆ
More informationJohns Hopkins University Department of Biostatistics Math Review for Introductory Courses
Johs Hopks Uverst Departmet of Bostatstcs Math Revew for Itroductor Courses Ratoale Bostatstcs courses wll rel o some fudametal mathematcal relatoshps, fuctos ad otato. The purpose of ths Math Revew s
More informationSTA 105-M BASIC STATISTICS (This is a multiple choice paper.)
DCDM BUSINESS SCHOOL September Mock Eamatos STA 0-M BASIC STATISTICS (Ths s a multple choce paper.) Tme: hours 0 mutes INSTRUCTIONS TO CANDIDATES Do ot ope ths questo paper utl you have bee told to do
More informationSimple Linear Regression and Correlation. Applied Statistics and Probability for Engineers. Chapter 11 Simple Linear Regression and Correlation
4//6 Appled Statstcs ad Probablty for Egeers Sth Edto Douglas C. Motgomery George C. Ruger Chapter Smple Lear Regresso ad Correlato CHAPTER OUTLINE Smple Lear Regresso ad Correlato - Emprcal Models -8
More informationTHE ROYAL STATISTICAL SOCIETY GRADUATE DIPLOMA
THE ROYAL STATISTICAL SOCIETY EXAMINATIONS SOLUTIONS GRADUATE DIPLOMA PAPER II STATISTICAL THEORY & METHODS The Socety provdes these solutos to assst caddates preparg for the examatos future years ad for
More informationLecture 1: Introduction to Regression
Lecture : Itroducto to Regresso A Eample: Eplag State Homcde Rates What kds of varables mght we use to epla/predct state homcde rates? Let s cosder just oe predctor for ow: povert Igore omtted varables,
More informationDiscrete Mathematics and Probability Theory Fall 2016 Seshia and Walrand DIS 10b
CS 70 Dscrete Mathematcs ad Probablty Theory Fall 206 Sesha ad Walrad DIS 0b. Wll I Get My Package? Seaky delvery guy of some compay s out delverg packages to customers. Not oly does he had a radom package
More informationENGI 4421 Propagation of Error Page 8-01
ENGI 441 Propagato of Error Page 8-01 Propagato of Error [Navd Chapter 3; ot Devore] Ay realstc measuremet procedure cotas error. Ay calculatos based o that measuremet wll therefore also cota a error.
More informationChapter 10 Two Stage Sampling (Subsampling)
Chapter 0 To tage amplg (usamplg) I cluster samplg, all the elemets the selected clusters are surveyed oreover, the effcecy cluster samplg depeds o sze of the cluster As the sze creases, the effcecy decreases
More informationRegression. Linear Regression. A Simple Data Display. A Batch of Data. The Mean is 220. A Value of 474. STAT Handout Module 15 1 st of June 2009
STAT Hadout Module 5 st of Jue 9 Lear Regresso Regresso Joh D. Sork, M.D. Ph.D. Baltmore VA Medcal Ceter GRCC ad Uversty of Marylad School of Medce Claude D. Pepper Older Amercas Idepedece Ceter Reducg
More informationHandout #8. X\Y f(x) 0 1/16 1/ / /16 3/ / /16 3/16 0 3/ /16 1/16 1/8 g(y) 1/16 1/4 3/8 1/4 1/16 1
Hadout #8 Ttle: Foudatos of Ecoometrcs Course: Eco 367 Fall/05 Istructor: Dr. I-Mg Chu Lear Regresso Model So far we have focused mostly o the study of a sgle radom varable, ts correspodg theoretcal dstrbuto,
More informationCHAPTER VI Statistical Analysis of Experimental Data
Chapter VI Statstcal Aalyss of Expermetal Data CHAPTER VI Statstcal Aalyss of Expermetal Data Measuremets do ot lead to a uque value. Ths s a result of the multtude of errors (maly radom errors) that ca
More informationDescriptive Statistics
Page Techcal Math II Descrptve Statstcs Descrptve Statstcs Descrptve statstcs s the body of methods used to represet ad summarze sets of data. A descrpto of how a set of measuremets (for eample, people
More informationr y Simple Linear Regression How To Study Relation Between Two Quantitative Variables? Scatter Plot Pearson s Sample Correlation Correlation
Maatee Klled Correlato & Regreo How To Study Relato Betwee Two Quattatve Varable? Smple Lear Regreo 6.11 A Smple Regreo Problem 1 I there relato betwee umber of power boat the area ad umber of maatee klled?
More informationModule 7: Probability and Statistics
Lecture 4: Goodess of ft tests. Itroducto Module 7: Probablty ad Statstcs I the prevous two lectures, the cocepts, steps ad applcatos of Hypotheses testg were dscussed. Hypotheses testg may be used to
More informationLecture 1: Introduction to Regression
Lecture : Itroducto to Regresso A Eample: Eplag State Homcde Rates What kds of varables mght we use to epla/predct state homcde rates? Let s cosder just oe predctor for ow: povert Igore omtted varables,
More informationLecture Notes 2. The ability to manipulate matrices is critical in economics.
Lecture Notes. Revew of Matrces he ablt to mapulate matrces s crtcal ecoomcs.. Matr a rectagular arra of umbers, parameters, or varables placed rows ad colums. Matrces are assocated wth lear equatos. lemets
More informationSTK3100 and STK4100 Autumn 2017
SK3 ad SK4 Autum 7 Geeralzed lear models Part III Covers the followg materal from chaters 4 ad 5: Sectos 4..5, 4.3.5, 4.3.6, 4.4., 4.4., ad 4.4.3 Sectos 5.., 5.., ad 5.5. Ørulf Borga Deartmet of Mathematcs
More informationMEASURES OF DISPERSION
MEASURES OF DISPERSION Measure of Cetral Tedecy: Measures of Cetral Tedecy ad Dsperso ) Mathematcal Average: a) Arthmetc mea (A.M.) b) Geometrc mea (G.M.) c) Harmoc mea (H.M.) ) Averages of Posto: a) Meda
More informationThe Randomized Block Design
Statstcal Methods I (EXST 75) Page 131 A factoral s a way of eterg two or more treatmets to a aalyss. The descrpto of a factoral usually cludes a measure of sze, a by, 3 by 4, 6 by 3 by 4, by by, etc.
More informationSpecial Instructions / Useful Data
JAM 6 Set of all real umbers P A..d. B, p Posso Specal Istructos / Useful Data x,, :,,, x x Probablty of a evet A Idepedetly ad detcally dstrbuted Bomal dstrbuto wth parameters ad p Posso dstrbuto wth
More informationChapter 5. Curve fitting
Chapter 5 Curve ttg Assgmet please use ecell Gve the data elow use least squares regresso to t a a straght le a power equato c a saturato-growthrate equato ad d a paraola. Fd the r value ad justy whch
More informationLine Fitting and Regression
Marquette Uverst MSCS6 Le Fttg ad Regresso Dael B. Rowe, Ph.D. Professor Departmet of Mathematcs, Statstcs, ad Computer Scece Coprght 8 b Marquette Uverst Least Squares Regresso MSCS6 For LSR we have pots
More informationThe number of observed cases The number of parameters. ith case of the dichotomous dependent variable. the ith case of the jth parameter
LOGISTIC REGRESSION Notato Model Logstc regresso regresses a dchotomous depedet varable o a set of depedet varables. Several methods are mplemeted for selectg the depedet varables. The followg otato s
More informationReaction Time VS. Drug Percentage Subject Amount of Drug Times % Reaction Time in Seconds 1 Mary John Carl Sara William 5 4
CHAPTER Smple Lear Regreo EXAMPLE A expermet volvg fve ubject coducted to determe the relatohp betwee the percetage of a certa drug the bloodtream ad the legth of tme t take the ubject to react to a tmulu.
More informationLecture Notes Forecasting the process of estimating or predicting unknown situations
Lecture Notes. Ecoomc Forecastg. Forecastg the process of estmatg or predctg ukow stuatos Eample usuall ecoomsts predct future ecoomc varables Forecastg apples to a varet of data () tme seres data predctg
More informationhp calculators HP 30S Statistics Averages and Standard Deviations Average and Standard Deviation Practice Finding Averages and Standard Deviations
HP 30S Statstcs Averages ad Stadard Devatos Average ad Stadard Devato Practce Fdg Averages ad Stadard Devatos HP 30S Statstcs Averages ad Stadard Devatos Average ad stadard devato The HP 30S provdes several
More information